Robustness of Learning That Is Based on Covariance-Driven Synaptic Plasticity

It is widely believed that learning is due, at least in part, to long-lasting modifications of the strengths of synapses in the brain. Theoretical studies have shown that a family of synaptic plasticity rules, in which synaptic changes are driven by covariance, is particularly useful for many forms of learning, including associative memory, gradient estimation, and operant conditioning. Covariance-based plasticity is inherently sensitive. Even a slight mistuning of the parameters of a covariance-based plasticity rule is likely to result in substantial changes in synaptic efficacies. Therefore, the biological relevance of covariance-based plasticity models is questionable. Here, we study the effects of mistuning parameters of the plasticity rule in a decision making model in which synaptic plasticity is driven by the covariance of reward and neural activity. An exact covariance plasticity rule yields Herrnstein's matching law. We show that although the effect of slight mistuning of the plasticity rule on the synaptic efficacies is large, the behavioral effect is small. Thus, matching behavior is robust to mistuning of the parameters of the covariance-based plasticity rule. Furthermore, the mistuned covariance rule results in undermatching, which is consistent with experimentally observed behavior. These results substantiate the hypothesis that approximate covariance-based synaptic plasticity underlies operant conditioning. However, we show that the mistuning of the mean subtraction makes behavior sensitive to the mistuning of the properties of the decision making network. Thus, there is a tradeoff between the robustness of matching behavior to changes in the plasticity rule and its robustness to changes in the properties of the decision making network

Spatio-Temporal Credit Assignment in Neuronal Population Learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain

Spontaneous Prediction Error Generation in Schizophrenia

Goal-directed human behavior is enabled by hierarchically-organized neural systems that process executive commands associated with higher brain areas in response to sensory and motor signals from lower brain areas. Psychiatric diseases and psychotic conditions are postulated to involve disturbances in these hierarchical network interactions, but the mechanism for how aberrant disease signals are generated in networks, and a systems-level framework linking disease signals to specific psychiatric symptoms remains undetermined. In this study, we show that neural networks containing schizophrenia-like deficits can spontaneously generate uncompensated error signals with properties that explain psychiatric disease symptoms, including fictive perception, altered sense of self, and unpredictable behavior. To distinguish dysfunction at the behavioral versus network level, we monitored the interactive behavior of a humanoid robot driven by the network. Mild perturbations in network connectivity resulted in the spontaneous appearance of uncompensated prediction errors and altered interactions within the network without external changes in behavior, correlating to the fictive sensations and agency experienced by episodic disease patients. In contrast, more severe deficits resulted in unstable network dynamics resulting in overt changes in behavior similar to those observed in chronic disease patients. These findings demonstrate that prediction error disequilibrium may represent an intrinsic property of schizophrenic brain networks reporting the severity and variability of disease symptoms. Moreover, these results support a systems-level model for psychiatric disease that features the spontaneous generation of maladaptive signals in hierarchical neural networks

Spike Timing and Reliability in Cortical Pyramidal Neurons: Effects of EPSC Kinetics, Input Synchronization and Background Noise on Spike Timing

In vivo studies have shown that neurons in the neocortex can generate action potentials at high temporal precision. The mechanisms controlling timing and reliability of action potential generation in neocortical neurons, however, are still poorly understood. Here we investigated the temporal precision and reliability of spike firing in cortical layer V pyramidal cells at near-threshold membrane potentials. Timing and reliability of spike responses were a function of EPSC kinetics, temporal jitter of population excitatory inputs, and of background synaptic noise. We used somatic current injection to mimic population synaptic input events and measured spike probability and spike time precision (STP), the latter defined as the time window (Δt) holding 80% of response spikes. EPSC rise and decay times were varied over the known physiological spectrum. At spike threshold level, EPSC decay time had a stronger influence on STP than rise time. Generally, STP was highest (≤2.45 ms) in response to synchronous compounds of EPSCs with fast rise and decay kinetics. Compounds with slow EPSC kinetics (decay time constants>6 ms) triggered spikes at lower temporal precision (≥6.58 ms). We found an overall linear relationship between STP and spike delay. The difference in STP between fast and slow compound EPSCs could be reduced by incrementing the amplitude of slow compound EPSCs. The introduction of a temporal jitter to compound EPSCs had a comparatively small effect on STP, with a tenfold increase in jitter resulting in only a five fold decrease in STP. In the presence of simulated synaptic background activity, precisely timed spikes could still be induced by fast EPSCs, but not by slow EPSCs

A biologically plausible model of time-scale invariant interval timing

The temporal durations between events often exert a strong influence over behavior. The details of this influence have been extensively characterized in behavioral experiments in different animal species. A remarkable feature of the data collected in these experiments is that they are often time-scale invariant. This means that response measurements obtained under intervals of different durations coincide when plotted as functions of relative time. Here we describe a biologically plausible model of an interval timing device and show that it is consistent with time-scale invariant behavior over a substantial range of interval durations. The model consists of a set of bistable units that switch from one state to the other at random times. We first use an abstract formulation of the model to derive exact expressions for some key quantities and to demonstrate time-scale invariance for any range of interval durations. We then show how the model could be implemented in the nervous system through a generic and biologically plausible mechanism. In particular, we show that any system that can display noise-driven transitions from one stable state to another can be used to implement the timing device. Our work demonstrates that a biologically plausible model can qualitatively account for a large body of data and thus provides a link between the biology and behavior of interval timing

Optimization of Muscle Activity for Task-Level Goals Predicts Complex Changes in Limb Forces across Biomechanical Contexts

Optimality principles have been proposed as a general framework for understanding motor control in animals and humans largely based on their ability to predict general features movement in idealized motor tasks. However, generalizing these concepts past proof-of-principle to understand the neuromechanical transformation from task-level control to detailed execution-level muscle activity and forces during behaviorally-relevant motor tasks has proved difficult. In an unrestrained balance task in cats, we demonstrate that achieving task-level constraints center of mass forces and moments while minimizing control effort predicts detailed patterns of muscle activity and ground reaction forces in an anatomically-realistic musculoskeletal model. Whereas optimization is typically used to resolve redundancy at a single level of the motor hierarchy, we simultaneously resolved redundancy across both muscles and limbs and directly compared predictions to experimental measures across multiple perturbation directions that elicit different intra- and interlimb coordination patterns. Further, although some candidate task-level variables and cost functions generated indistinguishable predictions in a single biomechanical context, we identified a common optimization framework that could predict up to 48 experimental conditions per animal (n = 3) across both perturbation directions and different biomechanical contexts created by altering animals' postural configuration. Predictions were further improved by imposing experimentally-derived muscle synergy constraints, suggesting additional task variables or costs that may be relevant to the neural control of balance. These results suggested that reduced-dimension neural control mechanisms such as muscle synergies can achieve similar kinetics to the optimal solution, but with increased control effort (≈2×) compared to individual muscle control. Our results are consistent with the idea that hierarchical, task-level neural control mechanisms previously associated with voluntary tasks may also be used in automatic brainstem-mediated pathways for balance